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ABSTRACT 

A descriptive research project was conducted to investigate 
how 57 public school districts in the United States used data-inf ormed 
processes and data support tools like the Quality School Portfolio (QSP) to 
address key district questions. The research project was a partnership among 
the American Association of School Administrators, the National School Boards 
Foundation, and the National Center for Research on Evaluation, Standards, 
and Student Testing. The project used both survey and case study protocols 
(four case studies) to investigate school district data use. This paper 
discusses key themes identified in the study, which include focusing on data 
use to address student academic achievement and building the cultural, 
technical, and organizational capacities of school districts to use data. The 
paper also describes the challenges school districts face as they focus on 
data use. It makes recommendations to address these challenges and identifies 
questions for future research on data use to improve the decision-making 
process. (Contains 2 tables, 3 figures, and 11 references.) (SLD) 
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Abstract 

The Research Project was a descriptive Research Project that investigated how 57 
public school districts in the United States used data-informed processes and data support 
tools like the Quality School Portfolio (QSP) to address key district questions. The 
Research Project was a partnership among the American Association of School 
Administrators (AASA), the National School Boards Foundation (NSBF), and the 
National Center for Research on Evaluation, Standards, and Student Testing (CRESST). 
The Research Project used both survey and case study protocols to investigate district 
data use. This paper discusses key themes identified in the study, which include focusing 
on data use to address student academic achievement and building the cultural, technical, 
and organizational capacities of school districts to use data. This paper describes the 
challenges school districts face as they focus on data use, makes recommendations to 
address these challenges, and identifies questions for future research on data use to 
improve the decision-making process. 




Paee 2 3 



District Data-informed Decision Making 



4/16/2003 



Our goal this year was to improve writing. We developed writing 
interventions and utilized the state assessment rubrics. Our goal was to 
make sure that 95% of our students were proficient or above on at least 
two out of four of the writing samples. We used a representative sample of 
students. Two assessors independently assessed writing samples. If there 
was a discrepancy, a third expert assessor would assess the paper. We used 
QSP and a modification suggested by a senior researcher at CRESST to 
include the quarterly writing data in an irmovative system of assigning 
scores to years. We vis’aalized the data that demonstrated improvem_ent. 
Using the gauge chart of QSP, we demonstrated that our students almost 
reached the goal (Over 93% were proficient or above). 

Assistant Principal 
School District A 



Introduction 

This statement by the Assistant Principal at School District A exemplifies how 
schools use data and technology-rich data support tools like the Quality School Portfolio 
(QSP) to achieve measurable goals in student achievement. School District A was one 
participant in the District Data Use Research Project funded in partnership with the 
American Association for School Administrators (AASA), the National School Board 
Foundation (NSBF), and the National Center for Research on Evaluation Standards and 
Student Testing (CRESST). The purpose of this article is to describe the findings and 
interpretations of the Research Project. This article identifies key themes related to the 
ability of school districts to carry out data-informed district improvement. These themes 
focus on organizational data culture, data capacity, and technical data capacity. 



Project Background 

In 1999, AASA received a grant from the Department of Education to help school 
districts realize the potential of data-driven decision making in their schools and to 
enliaiice the way tliey use data to infcrm key strategic district questions. The grant money 



was used to fund the District Data Use Project, which involved two-day workshops for 
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school districts where they learned how to use data support tools in the context of their 
district improvement or strategic planning process. The use of a data support tool called 
the Quality School Portfolio (QSP) was integral to the workshops. QSP is a Windows- 
based application. Norm-referenced and criterion-referenced assessment data can be 
organized and visualized using QSP to inform key strategic district questions. 
Demographic and various student event data (e.g. attendance) can also be organized and 
visualized using QSP to inform key strategic district questions. School improvement 
teams can use QSP to create custom groups and combination groups in order to identify 
students who meet one or more selection criteria. QSP users can use the groupings to 
generate visualizations such as line graphs, stacked bar graphs, and scatter plots. A 
variety of crosstab tables and advanced tables can also be produced using QSP. These 
data visualizations aid data users in interpreting data to inform and answer key strategic 
district questions. 

. AASA and CRESST organized the Project workshops over a three-year period and 
provided staff development for 57 school districts across the country. Each school district 
participating in the Project assembled a data team that included a school board member, 
the superintendent, a district project coordinator, and a variety of district data users. Both 
AASA and CRESST technical staff provided telephone and online support in the use of 
QSP or other data support tools to school districts after they participated in the 
workshops. In addition to the workshops, AASA planned and hosted several conferences 
over the three-year period in which data team members from school districts participating 
in the Project met to share how they were using QSP in the context of their district 
improvement process. Also as part of the Project, NSBF provided training and support 
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materials focusing on data use and questions that school boards could use to develop 
district improvement strategic plans as well as to inform school district policy. 

Study Overview 

In 2002, AASA asked CRESST to plan and implement the District Data Use 
Research Project (the Research Project) to determine to what extent school districts 
address the following three research questions. 

1. How do school districts collect and organize data to answer key district 
questions? 

2. How do school districts use data to answer key district questions? 

3. How do school districts take action and present results based on data used 
to answer key district questions? 

The CRESST team developed a research framework based upon the work of John 
Bransford and Barbara Means of the IBM K-12 Consulting Group (Bransford et al., 
1998). The research framework included the three research questions (as noted 
previously), as well as question characteristics and performance levels for the three 
research questions. The CRESST team then used the framework to develop a research 
plan that included two protocols: a survey that included both selected response and open- 
ended questions and a case study that engaged four districts in an intensive interview 
process paralleling the survey design. 

Methodology 

Overview 

Survey. Researchers from CRESST designed surveys that included questions aligned 
to the three research questions. The surveys collected information about the systems that 
school districts have in place to collect, organize, and visualize data; tl'»c questions school 
districts address usmg data; and some of the effects and consequences of data use. 
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Surveys were tailored to specific members of the district data team including one survey 
designed specifically for the superintendent, one survey designed specifically for the 
district project coordinator, and one survey designed specifically for data users including 
school board members. The survey data was collected and imported into data support 
tools (Excel and QSP), and then visualized for interpretation by the CRESST researchers. 

Case Study. Four school districts identified by AASA and NSBF participated in the 
case study. These four school districts were all strong data users. The case study gathered 
information about data use by these school districts and the effects of data use in these 
school districts, including tracing effects down to the school and classroom levels, as 
appropriate. Participants in the case study were all users of QSP. The case study involved 
on-site interviews with data team members who were either primary or secondary users 
of QSP. Primary users are defined as those users who use QSP directly in the district 
decision-making process. Secondary users are defined as those users who use QSP to 
implement the district plan. The case study also involved the collection of artifacts from 
the school districts that were coded and analyzed using the research framework. 

Research Sample 

Survey. The sample for the survey study included data team members from all 
school districts staff-developed by AASA and NSBF who were using QSP and other data 
support tools (N= 57) at the time of the study. Within these school districts, respondents 
included the school district superintendent, district project coordinator or technology 
facilitator, and all members of the school district data team including school board 
members. 
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Case Study. School districts in the case study (N=4) were selected based on 
recommendations from AASA and NSBF. These school districts were recommended 
based on their innovative use of data and the willingness of each school district to 
volunteer for the study. For the purpose of this study, the school districts that participated 
in the case study will be identified in the following way. 

1 . School District A 

2. School District B 

3. School District C 

4. School District D 

Research Instrumentation 

Survey. Questions were organized around major ideas that aligned with the 
research questions for the study. Three surveys were developed to meet the specific roles 
of subgroups participating in the study including the school district superintendent, the 
district project coordinator or technology facilitator, and all members of the district data 
team including school board members. Survey questions used either of three formats: 
selected response with five distracters, with one of the choices always providing the 
opportunity for the respondent to indicate “Does not apply;” categorical responses such 
as yes or no responses; and open-ended responses. The surveys were reviewed and 
approved by the Institutional Review Board of UCLA. 

Case Study. The case study used interview questions developed by researchers at 
CRESST. The questions aligned with the three research questions of the study and were 
approved by the Institutional Review Board of UCLA. A series of individual on-site 
interviews were conducted with the school district data team including the following: the 
superintendent, the district project coordinator or technology facilitator, and other data 
users including school board members. Rubrics were developed that analyzed the 
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responses to determine the extent to which key areas of the research questions were 
addressed. These rubrics were built upon a framework of planning, implementation, and 
collection of results. Performance levels for each of the main areas of the rubric were 
developed by the research team and critiqued and refined by school district data teams. 

The research team for the case study consisted of staff from CRESST, AASA, and 
NSBF. The school district project coordinator determined the school district data team. 
Research Data Collection 

Survey. Surveys along with a Consent to Participate in the Survey form, which 
was approved by the Institutional Review Board of UCLA, were sent to all school 
districts that participated in the District Data Use Project. Self-addressed stamped 
envelopes were also included with the surveys. Follow up calls were made to the school 
districts on a weekly basis to make sure that they understood the process and could 
complete the surveys in a timely fashion. Upon completion, surveys were mailed to 
CRESST where the data was entered into Excel spreadsheets for analysis. Open-ended 
responses were also entered into Excel. 

Case Study. Interviews were conducted over a two-day period at each school 
district site. Interview responses were recorded on paper by the researcher during the 
interview. Interview data were then transcribed into Microsoft Word documents. These 
documents were returned in electronic and paper formats to each interviewee along with 
self-addressed stamped envelopes. The interviewee was given two weeks to review, edit, 
and return the documents to the CRESST offices. 

Data from these transcriptions were then entered as “Information Nuggets” into 
Excel. Researchers ceded these “Information Nuggets” based upon how they aligned with 
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the three research questions. For the purpose of this investigation, an information nugget 
represents a discrete statement of information or response to an interview question. 

The Information Nuggets and their codes were merged using Microsoft’ Word’s 
m-ail merge features into a label format that was printed to labels. The labels contained 
the Information Nugget along with the school district name, research question, 
characteristic, performance level, and type of data user. The labels were then affixed to a 
colored note card that aligned to the type of data user. See Figure 1 for a sample of the 
printed labels used for the analysis. 

Figure 1 : Sample of label creating by merging an Information Nugget with school 
district information. 



Dsts User: Assists nt Principsi 
North Penn 06/1 7/02 

Our school administration demonstrated 
that the “Oaught Oeing Good” Discipline 
Intervention played a role in the 
reduction ot disciplinary referrals at 
Pennfield and provided teachers with 
frequent (monthly) disciplinary reports. 
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Researchers created one poster for each research question. The poster included the 
research question, rubric characteristics and performance levels, and space to affix the 
colored note cards with the Information Nugget labels. 

Research Analysis 

Survey. The unit of analysis for the survey study was the school district. A system 
of weighted averages was used to fairly represent the participation of each of the school 
districts in the study because there were variable numbers of data users for each district. 
Weighted averages were used for the multiple data users within the data user category to 
represent selected response data for individual school districts. Additionally, overlap of 
some survey completion was identified for school district superintendents who also acted 
as the district project coordinators. 

Categorical responses for the school districts were included in Excel. In the case of 
categorical variables where a yes responses could be recorded, a criterion of 50% for a 
yes response to a categorical variable was used to determine whether a categorical 
response would be included as representative of the data users for a given school district 
in the study. 

Data were imported into Excel and QSP for analysis and data visualizations were 
created. Researchers constructed survey findings based upon these data visualizations. 
Data interpretations were aligned with the key research questions. Data from the open- 
ended questions of the survey were entered into Excel. The researchers coded the data to 
the research questions and then they coded the data to the characteristics and the 
perfonnance levels of the study. Pivot tables and charts were used to visualize this data 
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quantitatively. Researchers developed findings based on the open-ended data 
visualizations. The findings also included specific examples of open-ended responses that 
supported or did not support the survey open-ended findings. 

Case Study. Colored note cards containing Information Nugget labels were taped 
to one of the three posters. Each poster contained one of the three research questions. 
Each colored note card was taped to a poster based upon its alignment with a research 
question including the characteristics and performance levels associated with the research 
question. The completed posters with the colored note cards containing Information 
Nuggets allowed researchers to visualize how participants responded to each research 
question. These visualizations were used to develop the findings. 

Researchers also developed quantitative Excel Pivot charts and tables that 
visualized alignment of the responses to the research questions based on predefined 
characteristics and performance levels associated with the research questions. Specific 
examples from the information nuggets were used to inform the findings. 

Demographics of Participating School Districts 

Fifty-seven (57) school districts participated in the District Data Use Research 

Project. Of these 57 school districts, 63% returned surveys to CRESST (N=36). The 
school districts participating in the survey component of the Research Project 
demonstrated the following demographic characteristics: 

• Mostly suburban (54%) 

• Medium-sized: 1001-5000 students (34%) 

• The uirec majcr respondent groups were superintendents (N=26), district project 
coordinators (N=27), or data users (N=96) 

® The main groups of data users were school principals (N=23), teachers (N=21), 
and assistant superintendents (N=i2) 
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• 11 school districts returned surveys from all three major respondent groups 

• Respondents were relatively new to the Project and the use of data within their 
school districts with superintendents engaged with the Project for the longest 
average time of 16 months. 

• Over half of the responding school districts (N=22) reported having participated 
in the AASA two-day training workshops. 

Findings 

Several themes emerged based on the survey and case study findings. These 
themes focused on building a data-informed school district culture, building 
organizational capacity, building, and building technical capacity. A discussion of each of 
these themes follows. 

Theme 1: Building a Data-informed School District Culture: Supporting a culture of 
inquiry 

The Research Project found evidence that school districts that use data to inform 
decision-making support a culture of inquiry. For example, the superintendent of School 
District D, identified three elements that were critical to creating a culture of data- 
informed improvement: fostering a spirit of inquiry, valuing openness and transparency 
to data, and ensuring humility in the use of data. This study found evidence supporting a 
culture of inquiry within school districts. 

Evidence from the survey found that school districts support a culture of inquiry 
by encouraging data users to ask broad questions aligned to student achievement. Table 1 
demonstrates that as school districts build a data-informed culture, they focus on 
questions aligned to outcomes and processes related to student achievement. 
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Table 1 : Which of the following questions did you address using data? 



Key Question with District 
data focus 


Superintendent 
(Yes Response) 


District Project 
Coordinator 
(Yes Response) 


Data User 
(Yes Response) 


To what extent did students 
achieve standards this year? 


88.5% (N=23) 


85.2% (N=23) 


81.25% (N=26) 


To what extent did students 
achieve standards over the 
past several years? 


50%(N=13) 


66.7% (N=18) 


65.63% (N=21) 


How effective were district 
programs in helping students 
achieve standards? 


76.9% (N=20) 


51.9% (N=14) 


65.63% (N=21) 


To what extent did various 
academic groups achieve 
standards? 


65.4% (N=14) 


66.7% (N= 18) 


37.50% (N=12) 


How effective was the district 
in identifying students at 
academic risk? 


46.2% (N=12) 


51.9% (N=14) 


59.38% (N=19) 



N = School District 



The questions listed in Table 1 foster a spirit of inquiry by asking data users to 
reflect on key district questions related to student achievement and how data use might or 
might not be used to answer those questions. Table 1 demonstrates that data users from 
all three data team respondent types use data to address the questions. 50% or more of the 
respondents (in all but 2 instances) used data to inform the key district questions listed in 
Table 1. This high response rate (50% to 88.5%) indicates these school districts are 
building a data-informed school district culture by using data to answer key district 
questions. 

Responses to the open-ended questions of the survey demonstrated that school 
districts that are fostering a data-informed school district culture support a culture of 
inquiry and that the primary focus of their inquiry process is on student academic 
achievement. A sample open-ended response from a district project coordinator follows: 




We use data to identify those students achieving below grade level who 
have not made more than one years expected progress in a year's time, 
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those at or above grade level who have had a significant drop in 
achievement from one year to the next, and those achieving at or above 
grade level who would benefit from additional challenge. 



In addition to the open-ended survey responses, responses by the case study 
participants supported a culture of inquiry. This section reports how case study school 
districts used data support tools to foster a data-informed school district culture by 
addressing student achievement in the following ways; 

• Enhancement of traditional data-use patterns 

• Innovations in the use of data 

• Resolution of misconceptions 

Enhancement of traditional data-use patterns. The infusion of data support tools like 
QSP into the traditional data-informed work of school districts enhanced the ability of 
school districts to build a data-informed school district culture. Traditionally, school 
districts created reports on large-scale assessment data like the Pennsylvania state test. 
School districts enhanced their use of this data by importing the data into QSP in order to 
improve the reports that were provided to the schools. The technology coordinator from 
District B described how he uses QSP to enhance his ability to produce data-informed 
reports for the schools. 

QSP is used by our district. I use QSP to create reports on the PSSA, 

New Standards Assessment, and the Quarterly Assessments. I provide 
these reports to our schools in order to improve their ability to use data 
in support of their school improvement plans. 

School administrative staff also recognized how the QSP tool could be 
used to enhance systems that are currently in place. The assistant principal at a 
middle school in District A perceived how QSP could help better organize data 
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that currently resided in a district-level system. Her comment concerning a 
student information system called Pentamation follows. 



We have a district computer management system called Pentamation. 
This is administrative software. It is difficult to get data from this 
system that would be helpful to our school: Student names, IDs, 
attendance, and so on. It would be great if this system produced 
electronic files that could be imported into QSP. It would save us a 
great deal of time. 

These responses illustrate how school districts can build a data- 
informed school district culture by enhancing traditional data-use patterns, 
either by generating data-informed reports or by providing electronic access to 
data. 



Innovations in the use of data. School districts working to build a data-informed 
school district culture by supporting a culture of inquiry devised innovative ways to use 
student achievement data imported into QSP. District C collected assessment data from 
multiple sources. Central administrative staff developed an innovative system to bring the 
data from these multiple sources into QSP. Data users were better able to inform 
decisions using these multiple sources. The comment from an assistant superintendent 
from District C describes this initiative. 

We use multiple measures for assessment to diagnose student’s gaps as 
well as strengths. The QSP tool is effective in helping us collect and 
visualize data from these multiple assessments. 

Data support tools like QSP provided many opportunities for school 

districts to innovate in the ways that they used data to inform district questions. 

For example. School District A disaggregated school-based writing days in 
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ways that helped show how district schools achieved their goals in writing. 

School districts also used QSP to produce unique data reports that supported 
findings of school district projects, such as the Special Education Self Study in 
School District D. 

Resolution of misconceptions. When building a data-informed school district culture, 
data integrity and unbiased data interpretation is critical. Data support tools like Excel 
and QSP can help school districts dispel misconceptions that staff maintains with regard 
to student academic achievement. For example, the principal of School District D and 
her staff did an analysis to determine anomalies between student scores on ITBS and 
assignment of grade-level status in literacy. This principal used data displays from Excel 
to help dispel the erroneous idea held by teachers that Title I students were the cause of a 
drop in reading scores. The principal’s comments about resolving this misconception 
follow. 

The Excel spreadsheet showed how students in one class demonstrated 
improvement in reading achievement as measured by the Gates 
McGinnty Test even though they are Title I students or on Free and 
Reduced Lunch. I used this document with teachers to demonstrate that 
it is not necessarily Title 1 students or low SES students who “bring 
down reading scores. 

An assistant principal of a middle school in District A was able to use data support 
tools to help dispel pursuit of solutions to unsubstantiated hypotheses made by teachers 
within the school. When building a data-informed school district culture, school district 
personnel have to move from “gut feelings” to data-informed statements. Her comment 
below describes the importance of using data to dispel unsubstantiated teacher 
hypotheses. 
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Teachers and the union sometimes come to me with questions that 
begin: We have a feeling that. . . ” Now teachers know that the data 
team and I will always use data to either corroborate or not corroborate 
the feelings. Teachers know that we will always respond with data and 
evidence in support or not in support of various teacher/union 
hypotheses.” 

This assistant principal from School District A described a specific example of how 

an erroneous teacher hypothesis was debunked using data. Her comments follow. 

Some of the middle schools considered dropping school visits by 
prospective 6th grade students during the school year. I used a student 
survey to demonstrate that students reported their actual school visit 
was the most important factor in helping them to adapt to the new 
middle school environment. Based on the data I allowed the program 
that facilitated the school visits by students to continue. 

Findings from the Research Project support the concept that school districts use data 

support tools and data support a culture of inquiry that helps to dispel misconceptions or 

erroneous hypotheses within the context of inquiry. The case study provided examples of 

how a culture of inquiry allowed the school districts to support or not support guesses, 

hunches, or gut feelings that professional staff maintained about student academic 

achievement. 

Building Organizational Capacity 

Leadership. The study also found that leadership at all levels plays a key role in 
building the organizational capacity to systematically use of data within school districts. 
Leadership at the school board level is especially effective in promoting the system-wide 
use of data. School board leadership in the use of data produces system-wide 
opportunities to plan data systems, use data, and produce results that help the district 
achieve strategic goals. From interview responses, it is clear that school boards set 
expectations for data-driven decision making. For example, the school board in School 
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District C created an expectation that all proposals presented to them would be supported 

by data. This school board also developed the expectation that they receive quarterly 

data-informed reports about student improvement in academic achievement. A school 

board member from School District C provided the following comment concerning its 

commitment to support data-informed district improvement. 

"Administrative requests now can be characterized by the following 
questions by the Board: 

1) What data supports the need for this request? 

2) What plan do you have to use data to monitor the success of this 
program in achieving district goals?" 

Leadership at the administrative level is critical in supporting data use throughout a 
school district. A central administrator of School District A explained, “As a result of the 
frequent assessing correlate required by the district, the schools have woven a 
dependence on data into their culture.” To support this culture of data-driven decision 
making, the central administrative staff in School District A provided leadership in the 
use of data within the district. Administrative staff of School District A demonstrated 
leadership when they regularly used data to support proposals they brought before the 
school board. The superintendent of School District A demonstrated leadership when he 
created an accountability task force with members from the district and school 
professional staff The charge of this task force was to develop a data-informed 
accountability system. 

“One key purpose of the Accountability Task Force will be to develop 
a plan to focus on both measurable elements of student achievement 
(Pennsylvania State Standards Assessment, district-level tests) and 
other performance factors (attendance, drop-out rates, discipline 
referrals, ccurse-taking sequence, etc.) in order to provide a complete 
picture of the district’s performance.” 
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The vision to establish a data-driven task force that supports decision making 
followed by the action of creating the task force by the superintendent demonstrates 
leadership in the use of data at the administrative level. 

According to interview responses, the superintendent of School District D worked 
side-by-side with central administrative and school staff to enter local student reading 
assessment data so that it could be organized and interpreted for an upcoming school 
board meeting. The willingness to do what it takes to prepare the data even by the 
superintendent shows leadership at the administrative level and sends a strong message to 
all professional staff in the school district that the use of data by the school district is 
important and that all members of the school district are expected to have an active role 
in using data. 

Leadership at the school level within districts is critical in developing models that can 
then be used district- wide. For example, the assistant principal of a middle school in 
School District A conducted numerous school- wide data-informed initiatives to achieve 
school-wide writing improvement goals. She demonstrated leadership when she helped 
three teachers within her building plan, and then implement data-informed action 
research projects within their classrooms as part of their professional evaluation. As a 
result of this principal’s leadership, teachers and staff at her school experienced the 
effectiveness of using data to drive instruction; staff now has a concrete understanding of 
the expectation on the part of the school board and the administration for data-driven 
decision making. The model developed for data-informed action research projects at the 
school level vein expanded to other schools, helping staff at those schools to 
understand the effective use of data. 
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Staff Development. Staff development is a key characteristic for building the 
organizational capacity for school districts to effectively develop and use systems to 
collect and organize data to inform key district questions. In order to effectively collect, 
organize and visualize data in support of key district questions, the district needs to plan 
and implement effective staff development. Respondents responded to the following 
survey statements: Staff development provided by the school district has been helpful in 
using data and Staff development provided by AASA has been helpful in using data. 
Respondents used one of the following choices: Strongly Agree, Agree, Disagree, 
Strongly Disagree or Don’t Know selected response. Data was weighted and then 
averaged. Figure 2 shows district user perceptions concerning the effectiveness of both 
district and AASA staff development initiatives. 

Figure 2 

Staff development provided by the School District and AASA has been helpful in 
using data 
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Figure 2 shows school districts perceptions by all types of respondents concerning the 
success or lack of success of data use staff development initiatives provided by the school 
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district and by AASA. School Districts showed satisfaction with the staff development 
provided by both. 

While survey respondents were generally satisfied with staff development in the use 
of data, they were less confident concerning the capacity of their school districts to use 
data independently. Respondents responded to the following survey statement: Our 
district has the capacity to use data independently without outside support. Respondents 
selected from the following choices; Strongly Agree, Agree, Disagree, Strongly Disagree 
or Don’t Know selected response. 

Figure 3 

Our district has the capacity to use data independently without outside support. 

Capacity to Use Data Independentiy Perceptions of Respondents (Weighted Averages) 

/^t students 




From the perspective of all district persormel, well over half questioned the capacity 
of their districts to use data independently despite having participated in both AASA and 
local District Data Use staff development opportunities. 
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The Research Project found that organizational leadership at all levels (school board 
to teacher) and staff development are essential to build organization capacity for data use. 
It cannot be assumed, however, that those two factors alone will build organizational 
capacity to use data independently. 

The Role of Organizational Systems for Using Data. Organizatiorxal approaches 
to district improvement that use data-informed inquiry approaches to improvement are 
well supported. (Bernhardt, 1998; Shmoker; 1999). Senge (2000) describes how schools 
can use a business model of a data-informed systemic approach to produce school 
improvement results. Cromey (2000) describes how the work of the North Central 
Regional Education Laboratories (NCREL) was successful in helping school districts in 
Illinois and Wisconsin use data-informed and technology-rich processes to achieve 
strategic goals. 

This Research Project found that organizational systems for using data are essential 
for building organizational capacity. Case study school districts demonstrated several 
unique organizational systems for data use. For example, the democratic and data- 
informed district council model in School District D represents an organizational 
structure approved by the school board that supports the systemic use of data to support 
initiatives to address the needs of the school district. The district council consists of 
voting members from the schools and community and non-voting members from the 
school board. Members of the council bring data-informed pre-proposals before the 
council to address specific needs. The council then votes to determine whether the 
proposal should be developed fiilly. If the pre-proposal receives four council member 
votes, the team can then develop a full proposal to present before the council. The full 
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proposal must also provide data that supports the need for the proposal; data to monitor 
the intervention; and the data that is used to judge the success of the intervention. The 
superintendent takes proposals that win a majority vote to the school board for their 
consideration. The use of the district council as an organizational system helps build 
School District D’s organizational capacity to use data. 

The administration of School District C used an innovative model developed in Texas 
called the Instructional Focus model to infuse data-informed decision making into all of 
the schools within School District C. During the interview process, an assistant 
superintendent described how School District C plans to address student achievement 
through the use of the Instructional Focus model by establishing instructional foci based 
on student academic need. The comment made by the assistant superintendent 
exemplifies how School District C created an organizational system to build 
organizational capacity to use data in support of student achievement. 



Schools use the data from standardized assessments to develop one 
instructional target in reading, writing and mathematics. Each grade 
level team used data to establish an instructional target and developed 
local assessments to determine to what extent students achieved the 
target over the course of the school year. Students were then identified 
as needing enrichment, reinforcement, remediation, re-teaching, or 
tutorials. Grade level teams used local assessments and data to 
continuously monitor the success of individual students and then 
regroup as necessary. Schools used the local assessment data to assess 
grade-level and school achievement of the instructional targets. 

The Research Project found two models that school districts used to establish 
organizational systems to use data. These models are the broad use of data model and the 
deep and multi levs! mode!. 
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Broad use of data model. In the broad use of data model, school districts organize 
to collect data broadly across the district. School District B uses the broad use of data 
model when it collects New Standards Reference Exam data across all grade levels 
within the district. 

Large-scale assessment data was available to all districts in the case study. Collection, 

organization, and analysis of this kind of assessment data across the district is common in 

the broad use of data model.. For example. School District A collected reading data from 

elementary schools throughout the district. The comment that follows from the assistant 

superintendent typifies the broad collection of data across the district. 

“We administer the Integrated Performance Assessments (IP A), which 
are extensive reading and writing assessments that are administered four 
times a year in all of our elementary schools. The classroom teachers 
score these tests and the results are submitted and tabulated.” 

Deep AND MULTI-LEVEL USE OF DATA MODEL. In the deep and multi-level use of data 
model, data collection can also be deeper and more focused. School District D uses the 
deep and multi-level use of data model when it collects data across multiple levels of the 
organization as part of the Special Education Self-Study Project. School District A 
collected data within the context of an organizational system that was deep and engaged 
multiple levels of the school community. At each of its schools, the central administration 
established an Effective Schools Program that involved staff in the participation in 
various Effective Schools teams. One of the teams was the Frequent Monitoring Team 
that provided all of the other teams with data to support their work. The assistant 
principal described their Effective Schools Program at the middle school in the following 
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We are a pilot school for the Effective Schools Program. As a part of 
this program, teams studied how moving a study support program from 
the lunch period to eighth period was more effective for students. 

Again, the Frequent Monitoring Team used teacher survey data to 
demonstrate from a teacher perspective that the 8th period study 
support program was effective in helping students improve their focus 
on academic achievement. The survey used questions like: Is the 
library accessible? Did student behavior improve? Our team provided 
data supporting improvement in student behavior. Based on the survey 
our school made adjustments to the schedule in order to remedy some 
of the problems that were identified. 

This Research Project found that the theme of building organizational capacity is 
critical for school districts that want to use technology-rich and data-informed processes 
for school/di strict improvement. Leadership at all levels within the organization plays a 
key role in driving data informed improvement. Both internally and externally driven 
staff development are organizational characteristics that support data-informed 
improvement. Yet even with staff development support, school districts in the project 
felt that they did not have the capacity to work with data independently. Districts used 
two organizational models to support their data-informed initiatives. They used a Broad 
Use of Data Model that collected data across the district and they used a Multi-level 
Model of data collection that focused on a deep use of data at multiple levels within the 
organization and also in conjunction with a focused project. 

Theme 3: Building Technical Capacity 

Using data in the context of school and district improvement means that school 
districts need the technical capacity to collect, organize, and visualize data in ways that 
will support an inquiry process to improve student academic achievement. Traditional 
metlicds of collecting, organizings and visualizing data use paper, pencil, and highlighters 
to identify areas of strengths and weaknesses in charts provided by state boards of 
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education or testing companies. Question-driven improvement is often limited by these 
paper reports and the ability of staff to find patterns within these reports. 

Since the signing of the No Child Left Behind Law, data has become a main focus for 
this country’s schools (U.S. Department of Education, 2001). Technology plays a key 
role in helping school districts and schools look at and disaggregate data for determining 
student academic achievement at various levels within schools. Technology tools are 
becoming more commonplace and even necessary for schools and teachers who want to 
use data-informed processes to inform improvement in student achievement. (Baker, E. 
L., & Herman, J. L. in press; Mayer, R. E. 2001; Mitchell, D. 2001). Research supports 
the effectiveness of technology in collecting student assessment data. Lee and Eller 
(2001) reported on the effectiveness of a data support tool called the Quality School 
Portfolio (QSP) in helping schools use a data-informed process to achieve their school 
improvement goals. Of course, implementation of this technology must also be routinely 
reviewed and evaluated to ensure maximum usage and benefits (Baker & Herman; In 
Press). 

Technology data support tools like QSP allow educators to ask a variety of questions 
about student academic achievement and then use the point and click grouping and 
reporting functions of QSP to help inform the questions. The technical capacity to use 
QSP means that educators have the hardware necessary to use QSP software; the 
technical capacity to prepare data for import into QSP, import data into QSP, export data 
from QSP, and use the key grouping, report and goal setting functions of QSP. 
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The availability of technology-rich data support tools like QSP means that school 
districts enhance their ability to use a wide variety of data in support of district 
improvement. Table 2 identifies the percentage of school districts that include data 
related to both academic and non-academic areas in data support tools like QSP. 
Table 2; 

What data are currently in the decision-support tools such as QSP? 



Data Area 


% 


N 


Student background (demographics) 


92.6 


25 


Achievement or standard-based data from state 


81.5 


22 


Achievement or standard-based data from district 


77.8 


21 


Program participation - funded programs 


77.8 


21 


Student attendance 


70.4 


19 


Student grades 


48.1 


13 


Student behavior 


40.7 


11 


ACT/SAT data 


37 


10 


Program participation - special district or school initiatives 


37 


10 


School or district wide progress tests 


29.6 


8 


Student preschool participation 


25.9 


7 


Diagnostic or placement tests 


18.5 


5 


Curriculum embedded 


18.5 


5 


Transportation information 


18.5 


5 


Student post graduation plans/placement 


14.8 


4 


Student volunteer or service learning data 


11.1 


3 


Other 


3.7 


1 



Table 2 shows that school districts have some but not extensive electronic access to 
basic student demographic information and large-scale assessment data. The availability 
of electronic data enhances the ability of district staff to ask key questions that can be 
informed by the data. For example, 92.6% of respondents report that student 
demographic information is in data support tools such as QSP and 81.5% of the 
respondents report that they include achievement or standards-based data from the state 
into data support tools like QSP. The ACT/SAT percentage of only 37% reflects a lower 
percentage of high schools reporting use of QSP compared to the overall respondents. 
However, as indicated in Table 2, school districts report that they include less electronic 
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access to non-academic data and classroom level tests. In order to support a culture of 
inquiry, school districts must continue to build their technical capacity to use data by 
expanding access to data. 

All of the case study school districts collected and used large-scale student 
assessment data and to some extent local assessment data prior to participating in the 
Project. By participating in the 2-day AASA training, school district data teams learned 
how to use the data support tool QSP and other data support tools to more effectively 
collect, organize, and visualize their data to address key district questions. These 
technology tools enhanced the ability of school districts to use data. 

The Special Education Self-Study Project carried out in School District D is an 
example of how a district used a system-wide committee approach and data support tools 
to address a variety of issues related to student achievement. QSP played a key role in 
helping the committee collect, organize, and visualize data from a variety of sources in 
order to systematically study student achievement throughout the school district. The 
director of Special Education who led the project commented on the importance of 
building technical capacity by using data support tools such as QSP in analyzing student 
assessment data as follows. 

Even though we did not get the Data Point software that was a part of 
the National Study of School Evaluation Process (NESSE) due to its 
high cost, our ability to use QSP gave us the data analysis component 
of the NESSE process. We would have abandoned the NESSE process 
if we were not able to use QSP to collect data and do the data analyses 
- especially related to student academic achievement. I feel that the 
need to incorporate data analysis into the NESSE process was a key 
factor in using QSP. The need drove the use of QSP. 

This case study component of the Research Project found that the school districts 
incorporated the use of the QSP tool and otlier data support tool to build district technical 
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capacity by using two models: the data expert model and the distributed technical 
expertise model. 

Data expert model. In the data expert model, an individual who has expertise in the 
use of district data prepares and communicates a variety of reports of both a standard and 
requested type. The assistant superintendent of School District C actively hired a staff 
member that had both technical and statistical expertise. The data expert for School 
District C was well respected within the school district community for his ability to 
produce data reports to inform school-based initiatives. The comment that follows by the 
School District C data expert demonstrates his ability to work effectively with district 
staff. 



The availability of a data person such as myself at the district level is 
very important to provide our schools with local data in a timely 
fashion. We now need to build the capacity of our local schools to use 
data. 

The data experts in a school district also become the champions of data use within the 
district. They help to build the technical capacity by developing workshops on using data 
support tools. Through these workshops and the data use support provided by the data 
experts, school districts could use data in a timely fashion to inform decisions at both the 
school and the district levels. 

Distributed technical expertise model. In the distributed technical expertise 
model, school districts would delegate the responsibility for using data to the individual 
schools. For example. District C provided data reports to schools from the data expert but 
it also expected each of the schools to have a data team that would use data to develop 
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instructional targets and monitor student achievement in attaining the targets. School 
District A trained a data team at the school level to use data-informed processes for 
school improvement. Central administration from District A then built technical capacity 
by using these school pilot sites to help train the other schools in the use of data-informed 
school improvement. 

The most robust example of the distributed technical expertise model, can be found at 
School District A, which established Frequent Monitoring Correlate Teams within each 
of their schools as part of their Effective Schools initiative, The Frequent Monitoring 
Correlate Team at a school provided data reports to other school teams who were 
pursuing the achievement of specific data-informed goals. The assistant principal at one 
of the middle schools described the work of the Frequent Monitoring Correlate Team in 
the following way. 

There are seven other teams at the school that depend upon the data 
provided by the Frequent Monitoring Correlate Team. The Safe and 
Orderly team actually asked the Frequent Monitoring Correlate Team 
to help them develop a survey to determine staff and student safety 
knowledge. The team helped with the survey, data collection, 
visualization, and interpretation. 

The distributed technology expertise model was not used as frequently in the Research 
Project as the data expert model was used. School District C used both models in its plan 
to use data-informed improvement. The data expert in School District C recognized the 
need to distribute technical expertise to the schools because the Instructional Focus 
initiative of the school district was beginning to tax his ability to provide school-specific 
data reports to schools in a timely fashion. The Research Project found that districts who 
are new to the use of data tend to establish a centralized data expert within the school 
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district before they engage in distributing this technical capacity to schools. Building 
technical capacity requires districts to build support for data use through the data expert 
model before expanding technical capacity through the distributed technology expertise 
model. 

Challenges 

The process of accessing and using data is a complex, energy intensive, and difficult 
task. Researchers, led by Norm Webb from the Wisconsin Center for Educational 
Research (Mason, 2001), reported on the challenges that several Milwaukee public 
schools encountered in trying to build both technical and organizational capacities to use 
data in support of informing key strategic decisions (Mason, 2001). In her research with 
the Milwaukee public schools. Mason identified 6 challenges that school districts face in 
using data to support school improvement. 

1 . Cultivating the desire to transform data into knowledge; 

2. Focusing on a process for planned data use; 

3. Making a commitment to acquire data; 

4. Organizing data management; 

5. Developing analytical capacity; and 

6. Strategically applying information and results. 

As emphasized throughout this discussion, the infusion of data-informed decision 
making into school districts is a difficult task. School districts who participated in the 
Research Project achieved many successes. Yet despite the many successes, districts also 
faced many challenges in infusing data-informed processes into their organizations. This 
section discusses the challenges faced by school districts. The subsequent section 
discusses recommendations designed to address the challenges. 

Creatfing a CaSicare o? Sata-Informsd Improvement 
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One superintendent reflected on the components of the culture required within school 
organizations to nurture and enhance data-informed decision making. He characterized 
data-informed school districts as having three essential cultural elements; a spirit of 
inquiry, openness and transparency to data, and humility. 

Evidence of a spirit of inquiry is evidenced by the work of School District D in its 
inquiry into drop out rates in the school district and by the work of School District B in 
its efforts to explore student participation in AP classes. The energy and organizational 
focus required to maintain a system-wide focus on inquiry is enormous. Tt is easy for 
districts to relegate inquiry to the pursuit of trivial questions or only to some elements 
within the organization, or not at all. Strong inquiry initiatives such as the ones 
previously described in this article represent models that can be replicated by other 
school districts. 

Fostering and maintaining an openness and transparency to data is another challenge 
faced by school districts. The use of data-informed processes is not a panacea for 
guaranteed success. There are times when the data does not support district interventions 
or initiatives. School districts must be willing to present the data and information and also 
be ready to make mid-course corrections that may not be politically correct. School 
District C conducted a data-informed evaluation of three computer-based reading 
programs. Assessment data did not support the effectiveness of these programs in 
improving student reading achievement. Yet School District C supported a culture of 
openness and transparency to data and presented the findings to the community. 

The fmal challenge to school districts that use data to inform decision making 

is the value of humility. “He or she who controls the data controls the agenda.” Staff 
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members within educational organizations who work with data need to be humble in their 
approach to data and be willing to share data and information unselfishly to support the 
important work of school districts in accomplishing their strategic goals. The central 
administration expert at School District C was very unselfish in his work with school 
principals. He tirelessly shared data in easy to understand data visualizations that 
supported the important work of school principals. Yet, within organizations there are 
individuals who not as unselfish in sharing important data and information. For example, 
staff that is in charge of student information data may not be willing to share this data in 
ways that support the work of staff in the department of curriculum and instruction. 
Humility in the use of data is a very real challenge for school districts. 

Building Organizational Capacity 

Building the organizational capacity to use data to support data-informed decision 
making is an important challenge faced by school districts. Again, school districts need 
the organizational capacity to collect, organize, manage, visualize, analyze, and apply 
data to inform key district questions. Yet building the organizational capacity of school 
districts can be hampered by several key factors. Lack of leadership at the board or 
superintendent level can impede system-wide implementation of data-informed 
improvement. Evidence was not found for strong leadership at the Board or 
Superintendent level within School District A resulting in strong but sporadic 
implementation of data-informed processes within the district. To some extent, the data- 
informed decision-making model was isolated within one school in the district without 
the requisite leadership to disseminate the model to all schools within the district. 
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A second key factor that impedes the ability of school districts to build the 
organizational capacity to effectively use data is the lack of availability of high quality 
assessments aligned to the instructional targets and standards of the school district. Many 
of the school districts were forced to use norm-referenced tests in a criterion referenced 
way. Although school districts, worked diligently to derive subtest data from the 
standardized tests for the purpose of informing student progress on specific instructional 
targets or standards, the effort became less than satisfactory due to the misalignment 
between standardized subtest areas and state or district learning standards. The school 
districts are faced with an enormous challenge of finding, creating or adapting criterion- 
referenced tests that align with their local and state standards. There is also the technical 
challenge of developing data support tools that collect, organize and visualize the 
criterion-referenced test data to inform student achievement. 

Staff Development 

Within the domain of organizational capacity is the key function of staff 
development. School districts are faced with the challenge of providing staff with the 
staff development opportunities that help them use data to inform improvement at the 
school and district levels. The staff development needs to provide access to teaching and 
learning expertise and to provide ways to apply this expertise to data-informed school 
improvement. It also needs to provide participants with technical and statistical support. 
School districts within the study used a variety of approaches to staff development. All of 
the school districts participated in two-day training sessions with AASA staff School 
District A than used this training as a springboard to initiate data-informed improvement 
at one of its schools. The model involved establishing data support teams at each of the 




Page 34 



35 



District Data-informed Decision Making 



4/16/2003 



schools to be used in conjunction with an Effective Schools Initiative being used at each 
of the schools. The district then drew upon the expertise of staff at the model school to be 
used in staff development initiative with all of the principals within School District A. 
School District B designated specific data-informed staff development training days 
throughout the year in conjunction with data-informed initiatives within the school 
district for all schools. School District B also used a model of staff developing and using 
data teams at each of its schools. 

While each of the school districts used a model for staff developing school staff in 
using data-informed processes within the context of district and school initiatives, there 
still remained the need for just-in-time technical and organizational support for district 
and school staff Access to a senior researcher from CRESST and a data expert from 
AASA provided all teams with the just-in-time support that school districts required to 
move forward on data-informed initiatives. 

The challenge that school districts face is the need to provide data-informed district- 
wide staff development to support the use of data and data support tools like QSP. It is 
also critical for school districts to provide specialized staff development support in 
technology, school improvement, statistics and the use of data, curriculum, and 
pedagogy. Finally, districts must have the capacity to provide just-in-time support to staff 
that are engaged in implementing data-informed district and school improvement 
projects. 

Communication 

Within the domain of organizational capacity is the key function of communication. 
School districts in the project used a variety of technology-rich and traditional means to 
communicate data-informed results and conclusions aligned to key district and school 
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improvement initiatives. School District D used a power point presentation to share the 
data-informed findings and conclusions of the Special Education self-study project. 
School District C used their Web site to share important student demographic and 
achievement data. School District B used a “Walk Through” communication venue that 
enhanced the ability of schools to share their student-based and data-inforaied work on 
the achievement of district goals. 

While the districts of the Research Project demonstrated many diverse 
communication venues for sharing data-informed results related to district and school 
initiatives, there still remains a challenge to refine and enhance these communication 
venues. For example, the work at the middle school in School District A needs to be 
communicated more effectively to the other schools within the district. While School 
District C, effectively communicated student academic achievement data on the Web, 
there needs to be enhanced efforts to use the Web to communicate data-informed results 
and findings of specific district and school improvement projects. Accompanying the 
communication of project results should also include ways to celebrate and publicly 
reward those data-informed efforts to improve the schools and school districts. 

Building Technical Capacity 

School districts interested in using data-informed decision making face the challenge 
of building the technical capacity necessary to effectively use data to inform key district 
questions. There is no question that school districts require technology to collect, 
organize, and visualize data. School districts need the technical ability not only to collect 
and visualize data but also the technical ability to help manage, analyze and apply data to 
inform key questions. This project focused on helping school districts use the data 
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support tool called the Quality School Portfolio (QSP). Use of QSP requires that data be 
collected and formatted in ways that it can be imported into QSP. 

Probably the biggest technical challenge faced by school districts is the artificial 
hurdles placed in front of them by test companies. McGraw Hill provides school districts 
with a software tool called Clarity that allows schools to view their standardized test data 
in a variety of report formats. However, the system does not allow the export of data into 
data support tools like QSP. School districts who want to export data in a row and 
column fashion must purchase an additional $700 module from McGraw.Uill. School 
District D faced similar challenges in trying to export Gates McGinnity data for import 
into QSP. The superintendent worked with administrative and school staff to manually 
enter this data into QSP. 

Student achievement data provided by the state also challenged the ability of school 
districts to accurately import data into their data support tools. The state of New Mexico 
changed its state test from the Terra Nova to the California Achievement Test (CAT). 
This change made it very difficult for school districts to align historical assessment data 
from the Terra Nova with the new CAT assessment data. Also the electronic formats used 
by states to store achievement data were problematic for school districts in the study 
especially the inaccuracies found in student identification numbers. 

The technical ability of staff in working with all of the features of the Quality School 
Portfolio was a challenge for some district staff For example, the assistant principal at a 
middle school in School District A wanted to use a line graph to visualize student 
perfoimancc cn 4 separate writing assessments administered throughout the year. The 
line graph reporting function of QSP would only allow for the visualization of student 
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achievement data on a yearly basis. This QSP feature initially stymied the assistant 
principal in using QSP to visualize the data. With the help of a senior researcher at 
CRESST, she found a way to visualize term data. 

A variety of data support tools existed in most of the districts including QSP. The 
integration of data from these multiple support tools to inform key district questions was 
a challenge for most school districts. For example, most updated student information data 
resided in SASl software at School District C. Student academic achievement data 
resided in the Clarity software of McGrawHill, and the writing assessment data resided 
in QSP. Integration of data from these multiple data support tools proved to be a 
significant challenge at School District C. 

Challenges Summary 

The success of school districts in planning, implementing, and producing results aligned 
to school and district improvement initiatives have been extensively described in this 
article. Yet despite these impressive results, school districts face significant challenges in 
planning, implementing, and collecting results aligned to data-informed change. This 
section of the report highlighted the following challenges faced by school districts. 

The ability of school districts to; 

■ Build and maintain a culture of data-informed improvement that includes the key 
values of spirit of inquiry, openness and transparency to data, and humility. 

■ Build and maintain the technical capacity to use data effectively in support of 
school and district initiatives. 

■ Build and maintain the organizational capacity to use data in support of school 
and district initiatives. 

“ Plan, implement, and evaluate effective staff development for using data- 
informed processes in support of school and district improvement. 

■ Improve and enhance communication venues for sharing data-informed initiatives 
and their results 
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Recommendations 

Based upon the findings of this Research Project and the challenges identified, this 
section of the discussion identifies key recommendations aligned to the key research 
questions of the study and the challenges discussed previously. 

Creating a data-informed culture 

• Recommendation 1 : Engage and inform school boards about the importance of 
data-informed and question-driven approaches to district improvement. 

• Recommendation 2: Establish board level expectations that proposals concerning 
strategic initiatives and policy include a strong foundation in data. 

• Recommendation 3: Promote a district climate and culture that fosters as spirit of 
inquiry, openness and transparency to data, and humility. 

Building the school district capacity to use data-informed processes 

• Recommendation 1 : Insure strong data-informed and question driven elements 
into the district/school improvement process. 

• Recommendation 2: Develop a 5-year plan for infusing data-informed and 
question-driven processes into district operations that begins with a focus on 
student performance and academic achievement. 

• Recommendation 3: Build and maintain the organizational capacity to carry out 
data-informed inquiry within the school district. 

• Recommendation 4: Build and maintain the technical capacity to carry our data- 
informed inquiry within the school district. 

Staff Development: 

• Recommendation 1: Establish data support teams at the district and school levels. 

• Recommendation 2: Provide ongoing district-level and school-level staff 
development support for using data support tools, statistical tools, and teaching 
and learning opportunities to achieve specific district and school strategic goals. 

• Recommendation 3: Provide just-in-time staff development support for district- 
level and school-level teams engaged in district and school-level improvement 
projects. 

• Recommendation 4: Develop online opportunities to staff develop district and 
school staff in the use of data-informed processes for school and district 
improvement. 

Communication: 

• Recommendation 1 : Plan and implement regular communication opportimities 
with key stakeholders to share products and results of data-informed school and 
district initiatives. 

• Recommendation 2: Plan and implement both technology-rich aiid traditional 
opportimities to share data-informed district/school improvement results. 

• Recommendation 3: Plan and implement a process for celebrating and rewarding 
the successful use of data to inform district and school improvement. 
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Future Research Questions 

Based upon the findings and discussion of this report, the following research questions 
would prove useful for future investigation. 

• How do school districts continue to build and sustain the technical and 
organizational capacities to use data-informed decision making? 

• How do districts move from beginning levels of data-informed decision making to 
expert levels of data-informed decision-making? 

• What are the key factors that contribute to the system-wide use of data to inform 
decisions at all levels within educational organizations? 

• Does a data-informed decision-making process within educational organizations 
result in improvement in student achievement? 

• To what extent does a focus on data-informed decision making detract from other 
important goals and responsibilities of educational organizations? 

• How can educational organizations gain access to and use high quality 
assessments that are aligned with key instructional targets? 

• What are the most effective staff development, technical and organizational 
support, and communication models to help implement data-informed decision- 
making within school districts 



Summary 

This paper discussed the findings of the District Data Use Research Project. Building the 
cultural, organizational, and technical capacities of school districts to effectively use 
technology and data-informed processes for improvement requires significant energy. 
School districts that participated in the District Data Use Research Project demonstrated 
measurable results in achieving district improvement. They also faced significant 
challenges in using data-informed processes for improvement. The assistant 
superintendent of School District C of the Research Project summarized how data can 
support improvement when he made the following statement; 



“We have created a data-informed instructional process that 
includes: looking at data, disaggregation of data, creation of 
instructional foci by grade and content based upon data, 
creation of timelines in which insti uction occur, continuous 
assessment and data collection, creation of a class profile and 
planned interventions (enrichjnent, tutorial, maintenance, and 
re-teaching), and reflection about the process.” 
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